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i.  abstract 
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I.  INTRODUCTION 


A.  Problem 


Since  the  inception  of  txie  Total  Ship  Simulation  Model 
Project,  the  SHIP  II  model  has  undergone  several  major 
revisions,  the  requirements  for  which  were  made  evident 
through  attempts  to  apply  the  model  to  the  solution  of 
a  variety  of  manpower  and  personnel  research  problems. 

The  result  is,  broadly  speaking,  a  third  ger.  ration,  large 
scale,  Monte  Carlo,  computer  simulation  mod?!  of  a 
destroyer  class  ship. 

There  is  a  growing  need  in  the  Navy  for  the  development 
and  use  of  manpower  planning  and  research  tools  such  as  the 
SHIP  II  model,  but  there  is  a  concomitant  increasing  concern 
with  the  technical  adequacy  and  cost  of  developing  and  using 
such  tools. 

The  problem  addres  d  in  this  report  has  been  accurately 
stated  in  a  previous  evaluation  of  the  SHIP  II  model: 

"In  the  field  of  simulation  technology  it  is  more 
common  than  not  to  fine  that  research  applications  of  computer 
simulation  models,  and  the  decisions  based  on  such  applications, 
preened  objective  evaluations  of  the  models  involved.  Further, 
many  models  used  for  prediction  and  design  evaluation  are 
never  subjected  to  objective  comparisons  with  alternative 
methods.  The  reasons  for  this  situation  are: 

•  Lack  of  an  adequate  model  evaluation  technology 

In  the  past,  social  sciences,  particularly  psychology, 
have  relied  heavily  on  statistical  methods  and  concepts 
originally  developed  in  the  biological  sciences.  'Social 
scientists  have  modified  and  expanded  the  techniques  and 
have  built  a  formidable  " armamentar ium"  for  research. 

Existing  techniques  are  being  applied  to  the  analysis  of 
outputs  of  simulation  models,  presumably  so  that  planners 
can  have  confidence  in  their  decisions.  The  questions 
of  model  validity,  reliability,  and  utility  are  rarely 
addressed  because  existing  techniques  are  unwieldy  for 
this  purpose. 
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.  Central  to  this  problem  of  the  lack  of  a  workable;,,  '  , 
evaluation  technology  is  the  fact  that  theoreticians  have,  /- 
failed  to  translate  general  model  characteristics  which 
Should  be  "considered"  during  evaluation  into  specific  * 
operational  criteria.  An  objective  evaluation  must  provide;  - 
for  the  measurement  of  the  characteristics  under  consideration 
One  obstacle  to  progress  has  been  the  theoreticians’ 
reluctance  to  scale  and  propound  units  of  measurements 
where  human  judgement  is  concerned.  The  research  consumer, 
on  the  other  hand,  is  intimately  familiar  with  the 
judgemental  aspects  but  does  not  have  the  technical 
knowledge,  incentive  or  the  time  to  set  forth  the 
considerations  in  the  required  fashion. 

a  Operational  constraints 

Typically,  once  the  modeling  concepts  have  been 
developed  to  provide  planners  with  the  categories  of 
information  they  need,  the  model  and  data  development 
efforts  are  completed  at  a  point  in  time  in  the  planning 
process  when  vital  decisions  must  be  made.  A  quick  "answer" 
must  be  generated.  Hence,  model  runs  are  made,  output 
data  analyses  are  conducted  and  there  is  not  enough  time  to 
answer  the  question,  "How  good  was  the  model  that  produced 
the  results?"  This  is  particularly  true  when  models  are 
developed  to  answer  specific  questions  concerning  specific 
systems  rather  than  to  handle  a  broad  class  of  problems. 

•Another  constraint  on  model  evaluation  is  cost. 

Given  that  available  funds  are  limited,  managers  are  more 
prone  to  devote  funds  to  efforts  which  result  in  a  visible 
product  —  answers  to  operational  questions,  than  to  allocate 
resources  to  a  less  obvious  product  —  assurance  that  the 
answer  is  accurate  and  that  the  technique  is  reliable. 

It  canr.ot  be  in  the  best  interest  of  the  naval 
research  and  development  community  to  use  untested 
simulation  models  in  the  planning  process,  regardless 
of  the  underlying  reasons,  simply  because  bad  decisions 
could  be  the  result.  A  bad  decision  is  one  that  leads  to: 

1.  excessive  system  cost  or  research  cost, 

2.  excessive  development  time,  or 

3.  poor  system  performance." 

(Ref.  8,  pp.  1“2) 
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It  is  the  purpose  of  this  report  to  review  all  prior 
research  applications  of  the  SHIP  II  model  and  to  synthesize 
past  findings  in  order  to  assess  the  model  from  the  stand- 
point  of  technical  adequacy  and  cost. 

C.  Background 

In  June  1967,  following  the  successful  application  of 
a  simulation  model,  COSIMO,  to  personnel  research  problems, 
the  Bureau  of  Nava]  Personnel  (Pers-A33)  initiated  exploratory 
development  research  (EDR) *  concerned  with  the  development 
of  a  Total  Ship  Simulation  Model  (TSSM)  •' 

The  purpose  of  the  research  was  to  test  the  feasibility  of 
developing  a  simulation  model  of  an  entire  ship  so  that 
personnel  research  could  be  performed  at  the  total  system 
level . 

The  EDR  was  completed  in  June  1968  and  was  reviewed  by 
the  advisory  group.**  The  results  of  the  review,  despite 
a  variety  of  cautions  concerning  cost,  scope  and  validity, 
were  generally  favorable  and  plans  were  made  to  continue 
the  research  into  the  advanced  development  phase  (Ref. 19). 
These  plans  provided  for  model  improvement,  data  refinement 
and  the  training  of  Navy  personnel  in  the  use  of  the 
TSSM  (Ref.  25) . 

In  November  1968,  a  Ship  Simulation  Research  Modeling 
Unit  was  formed  at  the  Personnel  Research  Laboratory  (PRL) . 

The  unit,  composed  of  two  analysts,  began  to  analyze  and 
evaluate  the  prototype  TSSM  (Refs.  19-23)  as  part  of 


*  fEe  Bureau  of  Naval  Personnel  '(Pers-A33;  contracted  with 
Serendipity,  Inc.  to  have  the  TSSM  developed.  At  the  same 
time,  an  advisory  group,  composed  of  representatives  of 
various  Navy  organizations  was  formed  to  provide  guidance 
to  the  effort. 

**  Additionally,  Serendipity,  Inc.  conducted  a  seminar  for 
potential  model  users  in  June  1968. 


the  larger  advanced  development  effort.*  The  purpose  of 
this  effort  was  to  qualify  (train)  three  Navy  Analysts  in 
all  aspects  of  simulation" and  to  refine  the  TSSM*  and  its 
data  base.**  The  Naval  Research  Laboratory  (NRL)  provided 
computer  time  and  services  (Control  Data  Corporation  3800 
computer) . 

In  January  1969,  the  Naval  Personnel  Research  and 
Development  Laboratory  (NAVPERSRANDLAB) ,  formerly  PRL, 

Ship  Simulation  Research  Modeling  Unit  relocated  to  the 
contractor's  facility  and  began  to  play  an  extremely 
complex  and  difficult  role.  They  were  responsible  for  the 
analysis  and  evaluation  of  the  TSSM  and  were,  at  the  same 
time,  students  of  the  contractor  in  simulation  modeling. 

The  technical  effort  was  realigned  in  January  1969  and  team 
training  was  primarily  focused  on  data  refinement  problems. 
Relatively  little  attention  was  devoted  to  TSSM  .characteristics 
and  experimental  uses  of  the  model. 

By  April  1969,  following  refinement  of  the  data  base 
the  first  analytic  study***  (Ref.  26  )  was  performed. 
Immediately  following  the  application,  the  model  was 
reprogrammed  and  the  second  generation  model  (SHIP  II) 
emerged.  SHIP  II  has  been  modified  and  improved  many  times 
since. 

In  May  1969,  a  set  of  functional  and  organizational 
concepts  for  the  conduct  of  simulation  research  at  the 
NAVPERSRANDLAB  was  formulated.  The  original  technical 
scope  of  modeling  research  at  NAVPERSRANDLAB  was  expanded. 

The  increased  scope  was  due  partially  to  the  desire  for  a 
center  of  capability  in  simulation  technology  on  the  part  of 
the  Bureau  of  Naval  Personnel.  Objectives  of  the  program 
were  expanded.'  The  emphasis)  however,  remained  Upon  the 
SHIP  II  model,  and  toward  the  end  of  1969,  when  SHIP  II 
had  been  improved,  it  was  recognized  that  the  basic  SHIP  II 
modeling  concepts  and  techniques  could  be  used  to  perform 
studies  of  a  variety  of  ship  types  and  that  the  model  could 
be  used  for  early  manpower  research  during  system  planning 
and  design. 


w  A‘3vahced~D eve lopmen t  Objective,  ADO'  43-07X.  '  ” 

**  The  TSSM  was  a  general  destroyer  model,  the  data  base 
was  from  a  DDG-2  class  ship. 

***  A  comparison  of  model  output  statistics  to  requirements 
set  forth  in  the  Ship  Manning  Document  (SMD)  for  the  DDG-2 
ci  "°s  ship. 
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However,  a  more  thorough  understanding  of  the 
statistical  characteristics  of  model  output  was  required 
before  regular  application  to  manpower  research  problems 
could  begin.  Subsequently,  a  number  of  studies  using 
SHIP  II  have  been  performed. 


KmJsammtamesxmsiK 


The  basic  purpose  of  all  of  these  studies  has  been 
the  exploration  of  model  sensitivity,  validity  and 
utility.  Each  of  the  studies  will  be  discussed  in  a 
subsequent  section  c~  this  report. 


As  mentioned  previously,  operational  constraints 
frequently  require  that  a  model  be  used  tc  answer  questions 
almost  immediately  after  it  is  completed,  but  before  it 
has  been  tested.  The  SHIP  II  development  and  evaluation 
effort  differs  from  most  modeling  efforts  in  two  respects. 

First,  SHIP  II  was  developed  as  a  general  manpower  research 
vehicle.  Many  variables  and  relationships  can  be  investi¬ 
gated  through  its  application.  With  data  changes,  the  model 
can  be  made  to  represent  a  variety  of  ship  classes. 

Because  the  model  was  developed  as  a  general  research  tool, 
the  demand  for  its  immediate  application  (outside  of  the 
context  of  model  research  and  development)  was  not  present. 
Second,  the  capability  under  development  at  the  NAVPERSRANDLAB 
encompasses  the  areas  of  techniques  research,  mpdel  development, 
and  applications  of  models  to  manpower  research.  The  develop¬ 
ment  and  application  of  an  appropriate  model  evaluation 
technology  which  considers  criteria  and  specific  techniques 
for  comparing  models  is  viewed  as  an  integral  part  of  the 
capability. 


Because  of  these  differences  it  has  been  possible  to 
plan  for  and  conduct  model  evaluations  prior  to,  or  at  least 
simultaneously  with,  its  application  on  specific  systems. 


II.  APPROACH 


A.  Methodology 

Modeling  technologists  and  the  consumers  of  manpower 
modeling  research  place  different  emphasis  on  the  criteria 
u§ed  in  evaluating  a  simulation  model. 

Research  consumers,  including  system  acquisition 
managers,  personnel  planners  and  system  designers,  are 
greatly  concerned  with  the  relatively  narrow  operational 
problems  which  the  use  of  a  model  can  help  solve.  Because 
of  this,  practical  criteria,  such  as  cost,  the  meaning  of 
results  for  their  purposes,  and  timeliness  of  output,  receive 
the  most  attention.  Modeling  specialists,  on  the  other 
hand  concern  themselves  more  with  the  technical  design 
characteristics  of  models,  e.g.,  programming  logic, 
output  reliability,  etc.  Naturally  both  the  modeling 
expert  and  the  research  consumer  must  consider  validity 
and  problem  relevance. 

The  essentials  of  a  sound  technical  evaluation  of  a 
simulation  model  are: 

•  criteria,  defined  in  operational  terms  and  which 
permit  measurement  and  quantification. 

m  a  procedure  for  combining  the  results  of  applications 
of  each  criterion  or  a  scheme  for  interpreting  evaluation 
results. 

•  sufficiently  detailed  information  describing  the 
model,  its  objectives  and  uses. 

B.  Procedures 

These  technical  requirements  were  satisfied  in  the  current 
evaluation,  viz. 

•  Criteria  were  established  for  each  application  study 
using  the  model.  They  included: 

a.  Validity 

b.  Reliability 

c.  Practicality 


Preceding  gage  Manii 
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These  criteria  are  defined  in  more  detail  for  each  of  the 
applications  discussed  in  this  report  (Ref.  6) . 

•  The  systematic  description  of  research  conducted 
using  SHIP  II,  combined  with  the  later  evaluative 
discussion  comprise  the  scheme  for  presenting  and 
interpreting  results. 

•  A  systematic  description  of  the  model  objectives  and 
technical  characteristics  was  available.  (The  overall 
SHIP  II  research,  development,  and  test  effort  is  shown 

in  Figure  1  (Ref.  9  and  10  )). 

C.  Data  Collection 

The  basic  approach  to  the  current  assessment  included 
the  following  steps: 

•  Review  of  all  SHIP  II  documentation 

•  Analysis  and  description  of  prior  research  and 
development 

•  Summary  evaluation  of  each  study 

•  Synthesis  of  results 

•  Documentation 


Ill .  RESULTS 


A.  Summary  Description  of  the  SHIP  SI 

A  comprehensive  description  of  SHIP  II  is  provided  in 
references  9  and  10  and  is  too  voluminous  to  be  incorporated 
directly  into  this  report.  However,  a  sufficient,  summary 
description  to  base  an  evaluation  upon  is  presented  in 
this  section. 

The  SKIP  II  model  was  developed  to  demonstrate  the 
feasibility  of  using  computer  simulation  models  for  a  broad 
spectrum  of  personnel  research  studies  including  the 
investigation  of  alternative  manning  concepts,,  maintenance 
and  work  policies,  training  concepts,  and  equipment  design 
characteristics.  SHIP  21  was  designed  to  reflect  realistically 
the  many  dynamic  and  complex  interrelationships  among  shipboard 
equipment,  personnel  and  operational  and  maintenance  task 
requirements  for  planned  and  existing  ships  in  the  fleet. 

The  model  was  originally  developed  using  a  guided 
missile  destroyer,  a  DDG-2  class  ship,  as  the  referent' 
system,  but  the  basic  model  concepts,  logic  and  techniques 
are  in  fact  independent  of  the  referent-,  system.  SHIP  II 
can  be  applied  to  studies  cn  a  variety  of  ship  types  which 
are  functionally  similar  to  the  QDG-2. 

Technical  Characteristics  of  the  Ship  Simulation  Model 

SHIP  II  is  an  event~oriented,  digital  model  which  uses 
the  Monte  Carlo  method  for  producing  .random  samples  of 
events  from  empirically  derived  frequency  distributions. 

At  the  beginning  of  each  simulation  run  a  list  of  events 
is  generated  and  placed  in  a  transaction  table  along  with 
the  time  of  event  occurrence.  Before  each  event  is 
initiated  a  subroutine  searches  the  transaction  table  to 
determine  the  particular  event  which  xs  fco  occur  and  advances 
the  clock  to  the  time  associated  with  that  event. 

Events  include  the  assignment  of  personnel  to  watches, 
maintenance  and  work,  equipment  failures,  training  exercises 
and  classes,  and  other  activities  and  occurrences  aboard 
ship.  The  model  is  driven  by  a  scenario  which  lists 
initiation  times  and  durations  for  basic  operational 
requirements  such  as  training  exercise,  changes  in  readiness 
conditions  and  evolutions. 


Preceding  page  blank 
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The  programs  are  written  in  FORTRAN  IV  and  currently 
operate  on  a  Control  Data  Corporation  3800  computer  at 
the  Naval  Research  Laboratory.  Because  of  the  size  of  the 
model  and  the  requirement  to  store  huge  quantities  of 
data  (over  150,000  unique  array  positions  are  required 
to  store  the  data  used  by  the  programs)  a  data  compression 
technique  known  as  bit-packing  was  employed  in  the  program. 
Additionally,  overlays  and  segments  are  employed  to 
accommodate  the  programs.  The  model  contains  one  hundred 
and  twenty-eight  subroutines.  Many  of  the  subroutines 
perform  the  basic  functional  simulation  of  such  events 
as  watch  assignment  and  training  exercises,  others  perform 
utility  functions  such  as  bit-packing,  input  and  output, 
while  still  others  perform  executive  functions  which 
control  the  initiation  of  program  subroutines. 

SHIP  II  is  modular?  individual  subroutines  can  be 
easily  removed  and  modified  without  extensive  reprogramming 
of  the  rest  of  the  model. 

One  interesting  feature  of  the  model  is  the  inclusion 
of  factors  called  parametric  variation  ratios,  which  can 
be  used  to  uniformly  alter  values  of  selected  data  categories, 
For  example,  if  the  value  of  one  of  these  factors  is  set 
to  1.0  no  change  will  occur,  if  set  to  0.8  a  20  percent 
reduction  in  the  specified  variable  will  occur  and  if  set 
to  1.2  a  20  percent  increase  will  result.  The  frequency 
distributions  employed  in  the  production  of  random 
events  such  as  equipment  failures  and  certain  tasks  can  be 
changed  easily. 

In  SHIP  II  the  input  variables  which  can  be  altered 
by  application  of  parametric  variation  ratios  are: 

Training  Session  Duration 

Hours/Week/Man 

Failure  Detection  Times  Under  Cond.  I 

Failure  Detection  Times  Under  Cond.  Ill 

Failure  Detection  Times  Under  Cond.  IV 

Failure  Detection  Times  Under  Cond.  V 

Failure  Detection  Probability  During  PM 

Operation  %  at  Cond.  I 

Operation  %  at  Cond.  Ill 

Operation  %  at  Cond.  IV 

Operation  %  at  Cond.  V 

Mean  Time  Between  Failure 
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Probability  of.  Degradation 
Deferrals  for  Assistant:  Prob» 
Defer, rals  for,  Assistant:  Mean 
Deferrals  for  Assistant:  Stan.  Dev. 
Deferrals  for  Parts:  Prob. 

Deferrals  for  Parts:  Mean 

Deferrals  for  Parts:  Stan.  Dev. 

Time  in  Function:  Mean 

Time  in  Function:  Stan.  Dev. 

Error  Probability :  Pi 
Error  Probability:  P2 
Error  Probability:  P3 
Error  Probability :  PT 
Delay  Times  (DT) 

Time  in  Function:  Mean 

Time  in  Function:  Stan.  Dev. 

Error  Probability:  PD 
Error  Probability:  P2 
Probability  -  PTR/PTR 
Time  in  Function:  Mean 

Time  in  Function:  Stan.  Dev. 

Manhours/Day 

%  Carryover  for  Ships  Work 


The  model  uses  the  exponential  distribution  for 
determining  equipment  failure  times  and  the  log  normal 
distribution  for  producing  corrective  maintenance  times. 

The  model  output  statistics  are  currently  compiled  on 
a  weekly  basis,  that  is,  each  week  serves  as  a  data  point 
for  all  output  variables.  However,  it  is  a  simple  matter 
to  alter  the  interval  of  time  over  which  data  are  compiled. 

For  any  given  simulation  application  the  model  can 
currently  handle  about  350  men  and  about  500  pieces  of 
equipment.  It  is  possible  to  accommodate  larger  manning 
and  equipment  sets  by  reallocation  of  core  memory  and 
reducing  the  numbers  of  other  variables  treated. 

The  SHIP  II  program  and  its  modifications  are  stored 
on  IBM  cards  and  tapes  and  input  data  are  also  on  cards 
and  tapes.  Model  outputs  can  take  the  form  of  tape, 
card,  or  hard  copy.  The  particular  nature  of  the 
application  dictates  what  medium  is  used  for  input  or 
output.  Running  time  for  the  model  is  about  40  minutes 
per  simulated  week  for  an  entire  ship  of  300  men,  500  equipment 


items,  and  about  20  mission  types.  This  is  reduqed^to 
about  3  minutes  of  computer  time  per  week  of 
wHen  simulating  a  single  division  of  about  20-  irieny 100-: 
pieces  of  equipment  and  the  same  number  of  mission  types . 
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Development  of  the  Model 

The  steps  in  the  development  of  SHIP  II  are  presented 
in  Figure  2.  The  specific  design  of  SHIP  was  based  on 
considerations  in  four  major  areas: 

•  factors  of  interest  to  Navy  planners, 

•  measures  of  system  performance, 

•  availability  of  sufficient  data,  and 

•  current  state-of-the-art  in  simulation 
programming. 

Determining  the  factors  of  interest  to  Navy  systems 
analvsts  involved  an  examination  of  the  questions  personnel 
planners  would  submit  for  solution  via  simulation. 

Measures  of  system  performance  were  chosen  that  would 
be  useful  in  examining  alternative  concepts  of  system  design 
and  planning.  The  primary  measures  selected  were  statistics 
reflecting  manpower  utilization  and  equipment  availability. 

Data  limitations  and  the  state-of-the-art  in  simulation 
programming  placed  dual  constraints  on  the  model  design. 

Data  problems  included: 

•  data  unavailability, 

•  unreliability  of  data  that  were  available,  and 

«  the  cost  of  data  acquisition  and  preprocessing 
for  input. 

State-of-the-art  problems  included  programming  limitations 
and  computer  memory  and  processing  time  constraints. 

Because  of  the  size  and  complexity  of  the  program,  simulation 
languages  such  as  SIMSCRIPT  were  deemed  inadvisable.  For 
this  reason  the  program  was  written  in  FORTRAN  IV. 


Steps  in  development  of  SHIP  II  model. 


After  the  general  model  design  characteristics  were 
determined,  the-  development  effort  centered  on  modeling 
specific  ship  functions-. 

The  first  problem  addressed  was  the  selection  of  functions.,.. 
Since  the  program  was  to  simulate  an  entire  ship  for  manpower 
research,  it  was  decided  that  every  shipboard  function  involving 
human  performance  was  to  be  treated.  These  functions  included 
the  performance  of  watch,  evolutions,  exercises,  division 
training,  and  the  various  classes  of  maintenance  and  ship's 
work. 


The  general  procedure  followed  in  modeling  ship  functions 
was  to  develop  a  logic  flow  chart  of  the  processes  and 
activities  taking  place  on  the  ship,  then  convert  that  into 
a  computer  program  flow  chart,  then  on  to  programming, 
keypunching  and  finally  debugging.  The  problems  of  model 
validity  and  verisimilitude  were  always  an  important 
consideration.  Since  some  of  the  intended  model  uses 
preclude  empirical  validation  of  results,  particular 
attention  had  to  be  devoted  to  ensuring  the  correspondence 
of  program  logic  to  real-world  processes.  Operational 
fleet  personnel  were  consulted  extensively  during  functional 
design.  There  were  two  factors  during  this  procejs  which  seemed 
at  cross-purposes;  compromises  had  to  be  made  between 
the  desire  for  face  validity  of  the  model  and  the  programming 
limitations.  In  cases  where  artifacts  were  introduced 
to  simplify  programming,  they  had  to  be  constructed  so 
as  not  to  introduce  variation  not  found  in  the  real-world. 
Real-world  processes  not  considered  relevant  to  the  model 
outputs  were  excluded. 


Output  Data 

The  selection  of  outputs  for  the  model  program  was 
dependent  on  two  major  considerations.  The  first  was 
the  potential  research  areas  for  which  the  model  would  be 
used.  The  output  variables  had  to  be  selected  carefully 
so  that  they  would  be  relevant  to  the  study  of  manning 
concepts,  work  policies  and  other  types  of  personnel 
research.  Second,  an  important  concern  was  the  susceptibility 
of  the  model  output  data  to  analysis.  Program  output 
reports  which  merely  recounted  the  events  that  had 
taken  place  during  the  simulation  would  not  necessarily 
be  useful  in  the  solution  of  the  problems  toward  which 
certain  model  runs  would  be  directed.  The  model  output 
reports  had  to  be  organized  in  such  a  way  that  relf;  /ant 


statistics  could  be  accumulated  and  analyzed.  It  was 
decided  that  output  data  would  be  accumulated  in  four 
major  categories: 

•  manpower  data, 

•  equipment  information, 

•  task  data,  ’ 

•  ships  readiness. 

The  manpower  data  comprise  the  hours  spent  on  the 
various  tasks,  organized  by  individual  man,  ship  organization, 
and  by  skill  category.  Equipment  data  include  operating  times, 
failure  data,  and  repair  information,  such  as  repair  time 
required,  and  repair  errors  made.  Task  data  include  tasks 
performed,  time  spent  on  various  tasks  and  tasks  remaining 
undone.  Readiness  data  include  total  ship  readiness  and 
the  contribution  of  each  mission  oriented  equipment  subsystem 
toward  ship's  readiness. 

Input  Data 

The  four  major  input  data  categories  for  the  model 
are  manning,  equipment,  tasks  and  ships  readiness.  Manning 
data  comprise  skills,  that  is,  skill  types  and  skill  levels? 
and  organization  —  the  shipboard  organization  of  men  into 
departments  and  divisions.  Equipment  data  include  the 
organization  of  equipments  into  subsystems  according  to  their 
mission  functions,  reliability  data,  and  maintainability 
information,  including  mean-time-to-repair  (with  variance) , 
the  requirements  for  special  troubleshooting  actions, 
and  the  frequency  of  maintenance  deferrals  for  assistance 
or  spare  parts.  Tasx  data  include  requirements  for  watch¬ 
standing,  ship  exercises,  evolutions,  division  training, 
corrective  maintenance,  planned  maintenance,  and  ships  work. 
Ships  readiness  data  include  readiness  rating  tables  for 
the  entire  ship,  and  for  each  equipment  subsystem  required 
for  mission  performance. 

The  level  of  detail  of  input  data  for  the  SHIP  II  model 
is  not  the  same  as  many  simulation  models  treating  detailed 
aspects  of  system  performance.  For  example,  using  simulation 
to  study  performance  in  a  combat  information  center,  one 
model  was  developed  which  treats  the  location  of  equipments 
within  the  CIC  and  the  performance  times  for  each  molecular 
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task  as  , discerning  , a  dot  bn  the  radar  scope^  reporting  !  ■  ’ 
the  location,  of  the  dot,  and  responding  to  that'  ihformation 
by  transmitting  a  weapons  control  command.  SHIP  II  models 
all  major  shipboard  functions  in  a  more  general. way. 

For  instance,  in  the  area  of  corrective  maintenance,  rather 
than  representing  the  performance  of  each  individual 
maintenance  task  to  the  part  level,  SHIP  II  treats  the 
maintenance  tasks  as  single  events  with  average  performance 
times  and  standard  deviations.  At  this  level  of  detail 
one  can  study  the  interactions  of  all  of  the  various  tasks 
taking  place  aboard  the  ship  and  the  interactions  of  the 
priority  system  that  orders  the  performance  of  these 
different  thsks. 
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Data  Sources 

As  illustrated  in  Figure  3,  the  input  data  set  can  be 
produced  by  one  of  two  methods,  the  choice  of  which  depends 
on  the  problem  to  which  the  model  is  being  addressed.  First, 
for  the  investigation  of  a  ship  or  shipboard  system  currently 
in  operation,  data  can  be  acquired  and  analyzed  from  a 
number  of  Navy  data  sources:  The  Maintenance  Material  Manage¬ 
ment  System  (known  as  3-M)  provides  maintenance  data  on 
the  corrective  maintenance  of  shipboard  equipments;  there 
is  a  Casualty  Reporting  System  (CASREPT)  which  also  provides 
data  on  shipboard  equipment  reliability,  especially  insofar 
as  they  pertain  to  the  readiness  of  the  ship;  the  Ship 
Manning  Document  provides  information  about  shipboard 
manpower  organization  and  about  ships  work  tasks,  that  is 
facilities  maintenance  and  administrative  tasks  which  are 
performed  on  a  day-to-day  basis;  the  planned  maintenance 
system,  specifically  the  maintenance  index  pages  (or  MIPS) 
provide  data  for  the  planncl  maintenance  requirements 
for  each  equipment  onboard  the  ship;  and  finally  the  deck- 
logs  of  ships  in  the  fleet  provide  information  about 
the  ships  operation,  i.e.,  training  exercises,  evolutions 
and  special  details,  which  are  used  to  generate  a  scenario 
of  operation. 


Second,  if  the  ship  being  investigated  is  not  yet  in 
the  fleet,  if  it  is  a  proposed  ship  or  a  ship  in  an  intermediate 
design  sta§e,  the  data  from  the  above  sources  may  not  be 
available.  Data  for  a  new  ship  can  be  generated  either 
from  similar  data  from  other  ships  or  from  data  from  the 
contractor  or  the  project  office.  These  latter  data 
may  be  approximations  or  estimates. 
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B.  Technical  Summaries  of  Previous  Applications 

Each  of  the  studies  conducted  using  the  SHIP  II 
model  will  be  described  in  this  section  according 
to  the  following  format: 

1 .  Background 

2.  Objective 

3.  Scope 

4 .  Method 

5 .  Results 

6 .  Evaluative  Comment 


I.  Comparison  of  the  Total  Ship  Simulation  Model 

Output  to~Ship  Manning  Document  (SMD)  ''Requirements 
(See  Ref  26) 

1 .  Background 

After  the  TSSM  had  been  completed  and  the  DDG-2 
data  base  revised,  the  need  for  model  logic  changes  was 
identified'  and  the  development  effort  was  continued. 

There  was  a  requirement  to  test  model  validity 
and  economic  feasibility  and  the  development  and  test 
schedules  could  not  be  satisfied  by  the  contractor. 
Consequently,  the  tests  that  were  conducted  did  not  involve 
the  model  and  data  that  comprise  the  latest  version  of  the 
model,  but  the  contractor  felt  the  basic  structure  and 
concepts  were  similar  enough  to  use  the  older  version  of 
TSSM  to  meet  the  testing  requirement. 

2.  Objective 

This  comparison  study  was  intended  to  demonstrate 
the  analytic  and  economic  feasibility  of  utilizing  the 
TSSM  (actually  TSSM  concepts)  for  manpower  planning 
research.  Of  specific  interest  was  the  "validity"  of 
workload  and  readiness  statistics  (validity  here  being 
defined  as  the  degree  to  which  the  model  outputs  conformed 
to  a  set  of  SMB  requirements) . 


3 .  Scope 


The  TSSM/SMD  comparison  was  based  on  DDG-2  data: 

•  All  enlisted  men  (319)  from  SMD 

•  Planned  Maintenance  (PM)  task  list  prepared 
from  MIP's  (800  jobs) 

•  Equipment  list  and  Corrective  Maintenance  (CM) 
data  prepared  from  Maintenance  Data  Collection 
System  (MDCS)  data  (400  equip,  items) 

•  Ship's  work  jobs  (127  tasks)  from  SMD 

•  Exercises  (34  types)  from  type  commander 
instructions 

4 .  Method 

a.  General  Approach 

The  approach  was  to  perform  a  simulation  of  a 
typical  period  of  operations  of  the  DDG-2  class  ship, 
and  to  compare  the  outputs  of  the  simulation  to  the 
data  in  the  SMD.  The  implicit  assumption  was  that  the 
SMD  data  would  reflect  the  real  world.  The  basis  for 
comparison  was  the  set  of  work  statistics:  manpower 
workload,  tasks  completed,  tasks  undone.  Equipment  statistics 
were  produced  in  the  runs,  but  did  not  contribute  to  the 
comparison  because  there  are  no  equipment  data  in  the  SMD. 

There  were  two  sets  of  comparisons:  simulation 
vs  SMD  and  run  vs  run.  The  simulation  vs  SMD  served  the 
primary  purpose  of  the  study?  the  run  vs  run  comparisons 
were  made  to  investigate  the  statistical  characteristics 
of  the  model.  Comparisons  were  based  on  mean  values  and 
graphs  of  time  series. 

b.  Study  Steps 

9  Data  collection  and  preparation  -  data  were 
collected  from  5  main  sources:  SMD,  maintenance 
index  pages,  MDCS,  type  commander  instructions, 
deck  logs.  Data  were  entered  on  prepared 
data  forms  and  key  punched. 
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o  Data  set  was  "debugged”  -  rid  of  errors  of 
format  and  consistency. 

•  Simulation  was  run  (3  runs)  for  debugging 
purposes . 

•  Errors  discovered  in  initial  runs  were 
corrected. 

•  Simulation  was  re-run  (2  runs)  for  data. 

•  Data  were  prepared  and  comparisons 
(simulation  vs  SMD  and  run  vs  run)  were 
made. 

5.  Results 

a.  The  data  collection  effort  was  extensive.  Data 
were  in  a  variety  of  forms  that  had  to  be  processed,  combined 
collated,  and  distilled  for  input  into  the  model. 

b.  The  preliminary  runs  disclosed  some  major 
flaws  in  the  model/data: 

•  Some  PM  jobs  with  long  times  were  in  a 
perpetual  queue. 

•  PM  data  did  not  account  for  multiple 
copies  of  equipments. 

•  Some  large  CM  times  selected  from  the  log¬ 
normal  distribution  caused  perpetual  queues 
for  CM. 

These  discrepancies  were  alleviated  by  subsequent  data 
and  program  changes. 

c.  The  final  runs  produced  data  that  were  compared 
to  the  SMD  data  and  to  each  other.  The  SMD  comparison 
showed  agreement  for  watchstander  workload,  less  agreement 
for  non-watchstander  workload.  The  non-watchstander 
discrepancy  was  primarily  in  the  areas  of  ship’s  work 
accomplished  (not  all  was  done)  and  maintenance  demands 
(based  on  MDCS  rather  than  PM/CM  ratio) i 


22 


6.  Evaluative  Comment 

a.  The  run-to-run  comparisons  demonstrated  the 
consistency  of  the  basic  model  and  showed  the  types  of 
data  that  the  model  produced  that  could  be  useful  for  in¬ 
vestigating  manning  problems.  They  were  primarily  quali¬ 
tative  -  although  the  numbers  were  tabulated  and  graphed, 
no  statistical  analyses  were  performed  on  the  values. 

b.  The  data  refinement  effort  was  extremely  time 
consuming. 

c.  The  version  of  the  model  used  had  sufficient 
deficiencies  to  warrant  reprogramming.  Because  of  the 
extensive  nature  of  the  changes,  the  analytic  conclusions 
derived  from  this  study  are  not  believed  to  pertain  to  any 
direct  evaluation  of  the  current  version  of  the  model 
(SHIP  II) . 

II.  DDG-2  Reduced  Manning  Study 
(See  Ref.  8,  Appendix  B) 

1 .  Background 

Working  with  the  Chief  of  Naval  Operations  (OP-100) 
and  NAVPERSRANDLAB  the  contractor  was  required  to  demon¬ 
strate  model  utility.  The  study  addressed  the  question, 
"What  are  the  effects  on  workload  distribution,  readiness, 
etc.,  of  reducing  the  manning  level  on  the  ship?". 

2.  Objective 

Purpose  was  to  compare  two  manning  levels  of 
DDG-2  to  demonstrate  how  the  model  could  be  used  in  man¬ 
power  research. 

3.  Scope 

This  study  was  based  on  the  full  set  of  DDG-2 
manning  data  and  scenarios  provided  by  OP-100. 

4 .  Method 

a.  Configure  model  with  standard  manning  set. 


b.  Run  a  typical  scenario#  collect  data. 

c.  Reconfigure  data  base  to  reflect  reduced .manning. 

d.  Run  with  same  scenario*  collect  data. 

e.  Compare  results,  determine  effects  of  reduced 

manning. 

5 .  Results 

When  manning  was  reduced#  individual  manhours 
increased#  ships  work  undone  increased,  amount  of  Division 
training  decreased  and  maintenance  jobs  undone  increased. 
Queues  were  more  numerous. 

6 .  Evaluative  Comment 

a.  The  study  demonstrated  potentially  useful  types 
of  applications. 

b.  The  a  priori  hypotheses  concerning  effects 
were  in  general  confirmed  following  a  statistical  analysis 
of  results. 


III.  DDG-2  Sensitivity  Analysis 

(See  Ref,  8) 

1 .  Background 

There  was  a  need  to  determine  if  selected  output 
variables  of  the  model  responded  to  changes  in  independent 
variables  associated  with  equipment  maintenance  and  ships 
work.  This  study  was  one  of  the  first  attempts  to  do 
sensitivity  testing  of  the  model. 

2.  Objective 

It  was  the  purpose  of  this  study  to  compare  responses 
of  15  output  variables  (including  planned  and  corrective 
maintenance#  manhours,  queues  and  length  of  time  in  queues, 
readiness,  cancelled  work,  etc.)  resulting  from  changes  in 
equipment  reliability  input  (mean  time  -between  failures) 
and  task  times  (Time-in-Function  (TIF) ) . 


3 .  Scope 

i 

The  study  wasbased  on  the  fullsetofDDG-2  data  ' 
used  in  previous  applications; 

4.  Method  ■'  r  - 

a.  Configure  model  with  standard  data  set ?  run 
standard  scenario  collect  data. 

b.  Make  input  changes  to  MTBF  and  conduct  runs  using 
same  scenario  for  each  level  of  MTBF  (MTBF  X  1.2,  MTBF  X  1.4 
and  MTBF  X  1.6),  collect  data. 

% 

c.  Make  input  changes  to  TIF  and  repeat  b.  for 
TIF  X  .625,  TIF  X  .714  and  TIF  X  .83.3,  cOliedt  dftta, 

d.  Analyze  results  using  Studentized  fiange  Test 

methods . 

e.  Determine  effects  on  each  of  the  15  output 
measures  of  altering  MTBF  and  TIF. 

5.  Results 

The  overall  impact  of  varying  both  the  TBF  and  the 
TIF  parameters  is  minor  in  light  of  the  changes  induced  in 
the  15  model  output  categories  studied.  Variability  of 
tempo  of  operation  appears  to  exceed  the  influence  of  changes, 
in  TBF  and  TIF.  The  result  of  varying  TIF  is  possibly  .  ‘ 

more  evident  when  presented  graphically,  but  none  of  the 
differences  between  BL,  TBF  and  TIF  conditions  were  statistically 
significant  at  the  0.1  level  when  tested  with  a  form  of 
studentized  range  test.  The  lack  of  practical  and/or 
statistical  differences  in  this  analysis  could  result  from 
a  model  which  is  insensitive,  a  poor  choice  of  parameters 
to  vary,  or  a  choice  of  unsuitable  output  data  categories 
to  study. 

The  amount  of  corrective  maintenance  work  done  during 
a  model  run  is  independent  of  the  other  ship  work  done, 
primarily  preventive  maintenance.  The  CM  workload  tends 
to  have  a  large  impact  on  the  workloads/work  patterns  of 
certain  critical  ratings  (e.g.,  ETs),  since  fluctuations 
in  equipment  failures,  sometimes  only  one  piece  of  equip¬ 
ment,  can  cause  exaggerated  fluctuations  in  weekly  workload 
output  statistics. 


The  system  readiness  measures  in  terms^of  percent  C-l 
may.  be  a  poor  measure,  in  general,  due  to  the* rigidity 
of  the  readiness  scoring  method.  In,  terms  of  .ship  operation, 
the  incidence  of  deferrals  seems  to>  overly  influence  the 
%  C-l  statistics  and  one  equipment  failure  may  degrade  a 
whole  subsystem,  even  if  there  is  some  form  of  back-up 
equipment  that  would  be  brought  into  service  under  actual 
fleet  operating  conditions. 

6 .  Evaluative  Comments 

a.  This  study  pointed  out  the  "masking  effects" 
of  logistics  problems.  That  is,  due  to  large  equipment 
downtimes  because  of  deferrals  for  parts,  readiness  degraded 
to  the  point  where  the  scheduled  scenario  events  could  not 
take  place.  In  terms  of  operational  requirements  the  effect 
seemed  realistic  but  did  not  allow  a  thorough  examination  of 
the  exclusive  effect  of  manipulating  the  independent 
variables.  Subsequent  analysis  limited  the  range  of 
possible  deferral  times. 

b.  The  finding  that  model  outputs  were  more  sensitive 
to  changes  in  tempo  of  operation  than  to  MTBF  or  TIF  changes 
agrees  with  expectations  and  increases  confidence  that  the 
model  behaves  realistically. 


IV.  SHIP  II  Sensitivity  Analysis 

(See  Ref.  14) 

1 .  Background 

All  previous  applications  of  the  model  had  been 
at  the  total  ship  level  and  analyses  had  dealt  with  a 
small  range  of  variation  in  independent  variables. 

The  need  for  more  quantitative  sophisticated 
evaluation  of  response  characteristics  of  SHIP  II  was 
recognized  along  with  the  requirement  to  demonstrate 
utility  to  potential  users  of  model  outputs. 

This  dual  purpose  study  was  designed  to  satisfy 
those  needs  simultaneously. 

2.  Objectives 

a.  To  test  the  applicability  of  SHIP  II  to  a 
smaller  data  set  with  greater  detail. 


fc.  To  investigate  the  sensxtlvrty  of-  aihjmtoer:;, ? 
of  output  measures  to  the  input,  variable,  r' :sc0Bar£o ^loadv,-' 


c.  To  aid  PMS  389  in  evaluating  a  specific 
proposed  manning  plar . 


3.  Scope 


The  study  concerned  the  ASW  division  of  the  DD-963. 
The  data  set  included: 


m 


f 


20  men  in  4  rating  groups 
17  watch  stations 

11  subsystems,  including  100  equipments 
113  PM  tasks 
11  Ship's  work  jobs 
5  types  of  training  exercises 


4 .  Method 


a.  Determine  scale  and  levels  for  the  independent 
variable  -  scenario  load. 


b;  Select  dependent  variables 

c.  Collect  input  data 

d.  Pilot  run  and  analysis 

e .  Study  runs 

f.  Output  data  collection/collation 

g.  Statistical  tests  and  analysis 

h.  Interpretation  of  results 

5.  Results 


a.  Mean  values  for  each  output  variable  in  each 
run  were  tabulated.  Variables  were: 
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•  Watch  hours,  individual  and  total 

•  Divisional  training  hours 

•  Evolution  hours 

•  PM  hours 

•  CM  hours 

•  Ship's  work  hours 

e  Total  hours 

•  CM  hours,  Rating/Rate  and  NEC 

•  PM  hours,  Rating/Rate  and  NEC 

•  Queues  for  Rating/Rate  and  NEC 

•  Interrupts  due  to  unavailable  Rating/Rate 
and  NEC 

•  Readiness  ratings 

•  PM  jobs  scheduled 

•  Ship's  work  undone 

•  Division  training  sessions 

b.  Mean  values  of  dependent  variables  from  different 
scenarios  were  compared  for  statistically  significant 
differences,  using  the  randomization  test  for  two 
independent  samples. 

c.  Linear  relationships  between  dependent  variables 
and  scenario  load  were  investigated  by  calculating 
regression  lines  and  correlation  coefficients. 

d.  In  a  special  (unscheduled)  study,  the  effects 
on  performance  of  a  reduction  in  maintenance  skill 

was  investigated.  The  number  of  men  for  the  two  treatment 
conditions  remained  the  same  but  the  subject  NEC  was  reduced. 
The  output  variables,  watchstanders  workload,  non-watchstander 
workload,  total  workload,  PM  jobs  cancelled,  and  ship's  work 
undone  were  studied.  It  was  found  that  a  reduction  in  number 
of  NEC's  was  feasible. 


6 .  Evaluative  Comments 


a.  The  study  demonstrated  that  a  number  of 

dependent  variables  responded  well  (i.e.,  were  sensitive) 
to  manipulation  of  the  independent1  variable  "sdenario 
load"  or  tempo  of  operations.  ‘ 

b.  The  conduct  of  this  study  aided 

N  AVPE RS RANDLAB  in  the  refinement  of  application  ■ 
procedures  and  had  the  effect  of  sharpening  insights 
for  future  application. 

\ 

c.  It  demonstrated  that  the  model  could  be  used 

to  handle  sub-optimization  (less  than  total  ship)  problems. 

d.  An  actual  consumer  found  results  useful  in 
manpower  planning  (for  the  DD-963  class) 


V.  DD-963  Baseline  Runs 

(See  Ref.  17) 

1 .  Background 

The  immediate  basic  technical  requirement  of  PMS 
389  had  been  satisfied  through  previous  applications  of 
SHIP  II.  PMS  389  wished  to  be  in  a  technical  position  to 
answer  later  questions  concerning  the  entire  ship  through 
use  of  the  model.  (The  previous  applications 
concerned  only  the  ASW  division.) 

2.  Objective 

The  DD-963  baseline  simulation  study  was  performed 
to  provide  a  base  of  output  data,  for  a  wartime  operating 
environment,  which  could  be  used  for  comparison  in 
performing  future  analyses. 

3 .  Scope 

The  data  set  for  the  tire  DD-963  was  used.  It 
included: 

•  232  men,  in  12  divisions 

e  214  watch  stations 
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•  36  subsystems,  including  373  equipments 

•  283  PM  tasks 

•  .  138  Ship's  work  tasks 

•  34  training  exercises 
4 .  Method 

Steps  were: 

a.  Collect  and  prepare  input  data 

b.  Run  simulation 

c.  Compute  means  and  standard  deviations  for 
variables  of  interest. 

• .  Results 

The  results  of  the  baseline  simulation  consisted 
of  a  set  of  output  statistics  that  could  be  used  as  a 
basis  for  future  studies.  In  addition,  they  could  be 
used  as  an  indication  of  expected  performance  given  the 
proposed  (baseline)  data  upon  which  the  simulation  was 
based. 

6 .  Evaluative  Comments 

a.  The  data  gathered  are  potentially  useful  to 

PMS  389. 

b.  This  study  made  a  marginal  contribution  to 
overall  model  evaluation. 

c.  Data  preparation  and  model  runs  were  performed 
more  quickly  and  inexpensively  *-han  for  previous 
applications o 


VI.  Intermediate  Evaluation  of  SHIP  II 
(See  Ref.  8) 

1 .  Background 

The  lack  of  adequate  techniques  for  systematic 
model  evaluation  was  recognized  within  NAVPERSRANDLAB 


(See  Introduction) .  The  simulation  research  effort  had 
progressed  to  the  point  where  overall  model  evaluation 
was  deemed  necessary  before  substantial  application 
could  begin.  .  . 

Criteria  needed  to  be  defined  and  propounded  and 
a  scheme  for  evaluation  developed. 

.  ,2v  Objectives 

The  objectives  of  the  SHIP  II  evaluation  study  were: 

a.  To  develop  a  methodology  for  model  evaluation. 

b.  To  evaluate  the  effectiveness  and  utility 
of  the  SHIP  II  model. 

3 .  Scope 

The  study  was  a  general  evaluation  of  SHIP  II 
ar.d  included  face  validity  evaluation  and  sensitivity 
analyses. 

4 .  Method 

a.  General  Approach 

A  generalized  methodology  for  model  evaluation 
was  developed.  The  methodology  was  broad  enough  to  cover 
other  types  of  models  as  well  as  the  SHIP  II  model.  Then 
the  methodology  was  followed  to  evaluate  various  technical 
characteristics  of  the  SHIP  II  model.  Model  data  for  the 
evaluation  were  available  from  pilot  runs  and  comparative 
analyses  that  had  been  performed. 

b.  Study  Steps 

•  Develop  evaluation  methodology 

•  Collect  and  tabulate  data  -  output  data  and 
data  about  the  model 

•  Analyze  SHIP  II  based  on  evaluative  criteria 
and  data. 


Criteria  included  validity#  reliability#  sensitivity, 
practicality,  relevance.  (Ref.  pp.  E-3  through  E-5 
for  Definitions'  or  Ref.  6)  ? 

5.  Results 

a.  A  comprehensive  description  of  SHIP  II  was 
prepared. 

b.  Results  of  applications  of  SHIP  II  were  tabulated. 

c.  An  evaluation  methodology  was  developed  and 
reported. 

d.  The  methodology  was  applied  to  SHIP  II,  and 
the  following  aspects  of  the  model  were  treated: 

Validity 

Reliability 

Sensitivity 

Practicality 

Relevance 

6 .  Evaluative  Comments 

a.  A  number  of  methodological  advances,  including 
a  single  index  of  model  value#  were  made. 

v 

b.  The  model  successfully  met  criteria  for  model 
goodness  when  certain  of  the  independent  variables  were 
manipulated  and  when  face  validity  was  examined. 

c.  The  overall  results  demonstrated  that  further 
work  with  SHIP  II  could  benefit  Navy  manpower  research 
and  planning. 


VII.  Two-factor  Study 
(See  Ref.  18) 

1 .  Background 

PMS  389  cooperated  with  NAVPERSRANDLAB  in  simulating 
DD-963  wartime  operations.  The  data 
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from  that  set  of  runs  was  considered  the  "standard" 

DD-963  data  set. 

Previous  applications  of  SHIP  II  had  focused  on 
the  relationships  between  output  (dependent)  variables 
and  single  input  (independent)  variables.  Yet  one  of  the 
major  reasons  for  using  simulation  modeling  is  to  study 
the  effects  of  interactions  between  independent  variables, 
which  are  not  easily  represented  analytically.  This 
study  was  the  first  attempt  to  consider  the  effects  of 
simultaneous  variations  of  SHIP  II  independent  variables. 

2.  Objective 

The  specific  problem  addressed  in  the  study  was  to 
explore  the  relative  effects  of  variations  in  manpower 
level  and  workload  demand  upon  workload  and  task  accomplish¬ 
ment. 

3 .  Scope 

The  study  was  run  on  the  ASW  division  of  the  DD-963 
class  ship.  Data  included: 

•  20  men  (this  number  was  varied  during  the  study) 

•  17  watch  stations 

•  11  subsystems,  including  100  equipments 

•  113  PM  tasks 

•  11  Ship's  work  jobs  (these  jobs  were  varied 
during  the  study) 

•  5  training  exercises 

4 .  Method 

There  were  two  independent  variables.  For  each 
variable,  five  levels  were  selected,  giving  25  combinations 
of  values.  For  each  of  the  25  treatment  conditions  a 
simulation  was  run,  using  a  combat  scenario  designed  by 
PMS  389. 

The  results  were  graphed  and  interpreted,  and 
were  analyzed  using  a  two  way,  fixed  effects  analysis  of 


variance  model*  Additionally,  the  relative  proportion 
of  the  variance  accounted  for  by  each  variable  was 
calculated. 

5.  Results 

The  following  were  the  14  dependent  variables 
of  interest: 

•  mean  watch  hours 

•  percentage  of  standard  Navy  work  week 

•  number  of  queues  (maintenance  manning) 

•  interrupts 

•  percentage  of  unavailability 

•  number  of  queues  (equipment) 

•  PM  delays 

•  PM  jobs  cancelled 

•  Ship's  work  hours  left 

ft  Ship's  work  interrupts 

•  Training  exercises  cancelled 

•  Training  delays 

•  Training  delay  hours 

The  workload  variation  consisted  of  a  systematic 
reduction  in  the  amount  of  ship's  work;  the  manning 
variation  was  a  systematic  reduction  in  the  number  of 
STG's.  Table  1*  summarizes  the  statistically  significant 
results.  Table  2*  summarizes  the  implications  of  data 
for  model  validity. 


*  Reprinted  from  Special  Research  Report:  A  Two-Factor 
Study  of  Manpower  Level  and  Work  Demand,  Schwartz  and 
Harris,  1.972 . 
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Results  Tend  Results  Tend 
to  Support  Not  to  More 

Model  Support  Model  Data 

Dependent  Variable  Independent  Variable  Validity  Validity  Needed 


In  addition  to  examining  the  specific  questions 
regarding  the  dependent  variables,  the  study  also 
addressed  some  general  topics  regarding  experimental 
design  and  the  application  of  the  SHIP  II  model.  The 
results  of  the  F-tests  .indicatedtthat  the  assumptions 
underlying  the  use  of  the  analysis  of  variance  (ANOVA) 
model  may  have  been  violated.  If  so,  it  seemed  probable 
that  the  specific  assumption  in  doubt  was  the  linearity 
of  the  data,  in  which  case  a  future  study  should  investigate 
possible  non-linear  transformations  of  the  data.  This 
could  be  an  important  result  in  terms  of  long  range  plans 
for  model  applications. 

6 .  Evaluative  Comments 

a.  A  new  and  useful  adjunct  to  the  SHIP  II  model 
was  developed  and  is  now  available  for  future  use  with 

the  model.  It  is  a  computer  program  called  "Linkage"  which 
selects  appropriate  variable  classes  and  prints  run  values 
for  them,  disregarding  non-relevant  data.  It  outputs 
directly  onto  punched  cards  which  are  then  used  in  various 
statistical  analyses.  Future  SHIP  II  studies  will  be 
greatly  expedited  as  a  result,  with  a  significant  reduction 
in  cost  and  time. 

b.  Many  of  the  responses  of  dependent  variables 
in  this  study  demonstrated  the  modelfe  construct  validity 
(when  workload  requirement  and  manning  levels  were  the 
independent  variables) . 

c.  The  practical  utility  of  the  model  for  the 
solution  of  several  classes  of  manpower  problems  was 
demonstrated . 


d.  There  were  minor  flaws  in  model  logic,  which, 
on  reexamination,  had  an  adverse  effect  on  equipment 
availability  output  measures.  These  will  be  corrected 
in  the  near  future. 
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VIII.  Reduced  Manning  Study  (Ref.  32) 

1.  Background 

The  possibility  of  an  all  voluntary  military 
force  has  caused  manpower  planners  to  consider  the  pro¬ 
spect  of  a  significantly  reduced  manpower  pool. 
NAVPERSRANDLAB  is  investigating  proposed  ways  of  meeting 
manning  requirements  with  such  a  reduced  personnel  pool. 

2.  Objectives 

a.  To  determine  the  effectiveness  and 
efficiency  of  using  SHIP  II  to  investigate  problems  of 
reduced  shipboard  manning. 

b.  To  develop  and  demonstrate  a  heuristic 
method  of  application. 

c.  To  assess  the  suitability  of  a  specific 
proposal  -  moving  tasks  ashore  -  to  reduce  shipborad 
manning  requirements. 

3 . .  Scope 

The  study  was  general  in  nature;  the  objectives 
were  not  limited  to  a  specific  ship  class  or  ether  data 
set.  However,  for  time  and  convenience  the  data  set  was 
limited  to  a  single  division  of  men  on  the  DD-963.  It  was 
assumed  that  study  results  would  be  generalizable  to  a 
total  ship. 


The  data  set  encompasses  the  S  division: 

©  38  men  in  5  ratings 

©  27  watch  stations 

•  149  ship's  work  tasks 

•  34  training  exercises 

®  a  one  year  scenario  of  wartime  operations 
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4 .  Method 


The  general  approach  was  a  three-part  iterative 

method. 


a.  A  baseline  run  to  establish  statistics  based 
on  currently  documented  manning  and  tasks. 

b.  A  reduced  task  run  to  establish  statistics 
based  on  full  manning  with  reduced  task  responsibilities. 

c.  A  manning  reduction  based  on  the  foregoing 
reduced  task  run,  followed  by  a  reduced  manning  run  to 
verify  the  results.  (If  the  reduction  criteria  were 
not  satisfied,  the  manning  reduction  would  be  adjusted 
accordingly,  and  the  reduced  manning  simulation  would  be 
re-run . ) _ 

5 .  Results 

a.  The  baseline  run  produced  a  set  of  statistics 
against  which  the  subsequent  run  results  would  be  measured. 

b.  Of  the  149  ship's  work  tasks,  64  tasks, 
totalling  316  manhours  per  week,  were  deleted  on  the 
assumption  that  they  might  be  performed  ashore.  The 
simulation  was  run  with  full  manning  and  reduced  tasks. 

c. '  On  the  basis  of  workload  statistics  generated 
in  the  reduced  task  run,  and  of  remaining  skill  and 
supervisory  requirements,  8  of  the  38  men  in  the  division 
were  deleted.  The  tasks  which  had  been  assigned  to 
those  men  and  had  not  been  eliminated  were  reassigned 

to  other  remaining  men.  The  simulation  was  then  run 
with  the  reduced  manning  to  test  the  efficacy  of  the 
manning  reduction. 

The  reduced  manning  set  proved  to  meet  the 
requirements  set  up  for  the  demonstration.  These 
success  criteria  were: 

«  all  watch  stations  must  be  manned  by 
division  personnel, 

©  tasks  could  be  performed  by  higher 

rated  personnel  than  originally  assigned, 
but  not  by  lower  rated  men, 
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•  average  workload  level  at  reduced  manning- 
reduced  tasks  should  approximate  the  level 
at  full  manning-full  tasks. 

The  study  showed  that  the  SHIP  II  model  was 
a  good  tool  to  use  to  investigate  this  type  of  problem. 

It  demonstrated  that  the  iterative  method  used  was  a 
sound  approach  to  investigating  the  problem.  Finally, 
it  showed  that  moving  ship's  work  tasks  ashore  might  be 
an  effective  way  to  reduce  shipboard  manning  requirements. 

6 .  Evaluative  Comments 

This  was  the  first  attempt  to  apply  the  model 
heuristically  as  a  planning  tool.  While  no  attempt 
to  quantify  the  degree  to  which  criteria  of  validity, 
etc.,  were  met,  it  is  felt  the  model  is  an  excellent 
preliminary  planning  tool  which  aids  in  problem 
identification  and  generates  data  which  form  the  basis 
for  the  creation  of  corrective  methods. 


IV.  DISCUSSION  AND  CONCLUSIONS 


While  the  basic  structure  of  the  SHIP  II  simulation 
model,  with  its  stochastic  processes  for  determining 
equipment  failure  times  and  task  completion  times,  and 
processes  for  establishing  man-to-task  assignments,  is 
not  difficult  to  understand,  the  number  of  manipulable 
and  interactive  variables  make  the  model  extremely  complex. 

The  task  of  comprehensive  evaluation  is  equally  complex. 

If  one  considers  the  requirements  necessary  to  evaluate 
response  characteristics  of  a  model  with  over  30  possible 
major*  variables  which  can  be  used  as  independent  variables 
and  a  like  number  of  major  dependent  variables,  one  is 
faced  with  a  huge  cost  and  time  problem.  Only  the  most 
important  variables  of  SHIP  II  can  be  considered  in  this 
evaluation  study  for  obvious  reasons.  In  general,  it 
can  be  said  that  SHIP  II  meets  the  criteria  for  face 
validity,  reliability,  practicality,  relevance  and 
sensitivity  for  those  combinations  of  variables  thus  far 
studied,  minor  flaws  in  data  and  logic  notwithstanding. 

The  point  has  been  made  that  for  the  ultimate  test 
of  validity  there  is  no  substitute  for  empirical  validity 
studies.  But  considering  the  expense  of  such  studies  and 
considering  the  fact  that  revalidation  might  be  required  every 
time  model  logic  is  altered,  it  may  not  in  fact  be 
particularly  cost  effective  to  conduct  such  studies. 

There  are  many  untested  features  of  SHIP  II  -  the 
test  and  evaluation  effort  should,  however,  be  guided  by 
specific  needs  for  applications  on  the  part  of  potential 
research  consumers. 

Past  applications  of  the  SHIP  II  model  have  neglected 
exhaustive  determinations  of  the  statistical  distribution 
characteristics  of  the  model  outputs.  This  could  present 
a  major  problem  where  interpretation  of  results  of  studies 
is  concerned.  Most  techniques  applied  to  analysis  of 
outputs  have  required  assumption  of  normal  or  linear 
distributions.  Whether  these  assumptions  were  true  of  the 
data  generated  by  the  model  has  not  yet  been  determined. 


Major  variables  refers  to  a  variable  which  can  be  considered 
a  set  of  smaller  components,  e.g.,  workload,  as  a  major 
variable,  can  be  comprised  of  watch,  PM,  CM,  etc.  for 
each  man,  each  division,  etc. 
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For  the  studies  thus  far  conducted,  SHIP  II  has.' 
been  shown  to  be  useful  in  the  following  kinds  of  manpower, 
problems : 

•  The  study  of  effects*  of  reducing  manpower, 
workload  requirements  or  both  simultaneously, 

•  The  study  of  the  effects  of  varying  ships 
tempo  of  operations, 

•  The  study  of  the  effects  of  varying  equipment 
reliability, 

•  The  heuristic  study  of  alternative  ways 
of  staffing  and  training  selected  ships 
functions , 

•  The  study  of  the  effects  of  altering  skill 
levels  of  personnel ,  ' 


•  Detailed  studies  of  workload  balancing  . 
problems , 

•  All  of  the  above  for  planned  or  actual, 
total  or  partial  systems. 

Currently  the  following  deficiencies  of  SHIP  II  are 
recognized: 

•  No  details  of  human  performance  are  accounted 
for 

•  Deficiencies  in  program  logic  (which  will 
be  corrected)  require  that  equipment 
availability  statistics  be  manually'  ' 
adjusted  prior  to  use  in  analysis  of  results 
of  studies. 

•  The  time  and  cost  involved  in  data  preparation 
are  significant.  This  is  true  of  most  modeling 
efforts . 


*  Effects  refer  to  effects  on  each  model  output 
statistic  previously  discussed. 


/ ' 
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•  In-port  conditions  are  not  simulated 


•  Persons  intimately  familiar  with  all  aspects 
of  the  model  must  be  connected  with  each 
application  of  SHIP  II  and  associated  programs. 
This  is  true  of  most  complex  models. 

©  The  model  in  its  current  form  can  only  be 
used  on  a  CDC  3800  computer. 

Application  costs  vary  with  the  details  of  the  study 
requirements  and  are  primarily  dictated  by  the  input  data 
collection  and  preprocessing  effort,  not  computer  time 
(approximately  $200  per  hour  and  an  average  of  $2506  per 
detailed  study) .  On  the  basis  of  past  experience, 
approximately  3  man  months  are  required  to  collect  and 
preprocess  a  whole  set  of  model  input  data  for  a  study. 

Output  processing  is  accomplished  by  computer  at 
minimal  cost  to  the  effort. 
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